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ADAPTIVELY ENCODING A PICTURE OF 
CONTRASTED COMPLEXITY HAVING NORMAL 
VIDEO AND NOISY VIDEO PORTIONS 



Technical Field 



5 This invention relates in general to compression 

of digital visual images, and more particularly, to a 
technique for encoding one or more frames of 
contrasted complexity within a video sequence using 
image statistics derived from the frame (s) to 
10 dynamically change one or more controllable encoding 
parameter (s) used in encoding the frame (s) . 



Background of the Invention 



Within the past decade, the advent of world-wide 
electronic communications systems has enhanced the 

15 way in which people can send and receive information. 
In particular, the capabilities of real-time video 
and audio systems have greatly improved in recent 
years. However, in order to provide services such as 
video -on -demand and video conferencing to 

2 0 subscribers, an enormous amount of network bandwidth 
is required. In fact, network bandwidth is often the 
main inhibitor in the effectiveness of such systems. 



In order to overcome the constraints imposed by 
networks, compression systems have emerged. These 
2 5 systems reduce the amount of video and audio data 

which must be transmitted by removing redundancy in 
the picture sequence. At the receiving end, the 
picture sequence is uncompressed and may be displayed 
in real-time. 
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One example of a video compression standard is 
the Moving Picture Experts Group ("MPEG") standard. 
Within the MPEG standard, video compression is 
defined both within a given picture and between 
5 pictures. Video compression within a picture is 

accomplished by conversion of the digital image from 
the time domain to the frequency domain by a discrete 
cosine transform, quantization, and variable length 
coding. Video compression between pictures is 
10 accomplished via a process referred to as motion 

estimation and compensation, in which a motion vector 
plus difference data is used to describe the 
translation of a set of picture elements (pels) from 
one picture to another. 



15 The ISO MPEG-2 standard specifies only the 

syntax of bitstream and semantics of the decoding 
process. The choice of coding parameters and trade- 
offs in performance versus complexity are left to the 
encoder developers. 



2 0 One aspect of the encoding process is 

compressing a digital video image into as small a 
bitstream as possible while still maintaining video 
detail and quality. The MPEG standard places 
limitations on the size of the bitstream, and 

25 requires that the encoder be able to perform the 

encoding process. Thus, simply optimizing the bit 
rate to maintain desired picture quality and detail 
can be difficult. 



A video picture typically contains both busy and 
3 0 simple macroblock segments, and there is a high 

correlation between the segments. However, certain 
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video frames are of highly contrasted complexity 
having, e.g., both normal video and noisy (or random) 
video portions within the frame, such as DIVA. 
Further, both the normal (or simple) video portion 
5 and the noisy portion are often moving from frame to 
frame. Within such a frame, most of the encode bits 
can be consumed by macroblocks of the noisy portion 
before picture coding is completed, thereby producing 
blockiness or artifacts within the picture and uneven 
10 output picture quality. 

This invention thus seeks to enhance picture 
quality of an encoded video sequence having one or 
more pictures with areas of significantly contrasted 
complexity, and more particularly, to enhance picture 
15 quality by dynamically balancing picture bit 

allocation as the picture coding continues without 
requiring lengthy buffering or high computational 
intelligence . 

Disclosure of the Invention 

2 0 Briefly summarized, the invention comprises in a 

first aspect a method for encoding a video frame 
having a noisy portion and a normal video portion. 
The method includes for each macroblock of the frame: 
determining a macroblock activity level; determining 
25 whether the macroblock activity level exceeds a 

predefined threshold, wherein the macroblock activity 
level exceeding the predefined threshold indicates 
that the macroblock is associated with the noisy 
portion of the video frame; and adjusting encoding of 

3 0 the macroblock when the macroblock activity level 

exceeds the threshold to conserve bits used in 
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encoding the macroblock and thereby reduce the number 
of bits used to encode macroblocks within the noisy 
portion of the video frame. 

In another aspect, a method is presented for 
5 encoding a frame of a sequence of video frames, each 
frame having a plurality of macroblocks. The method 
includes: determining whether the frame includes a 
random noise portion; and when the frame does include 
a random noise portion, evaluating each macroblock of 

10 the plurality of macroblocks in the frame and 

adjusting encoding of at least some macroblocks 
within the random noise portion of the frame, the 
adjusting of encoding comprising conserving bits used 
in encoding the at least some macroblocks within the 

15 random noise portion of the frame. 

In still another aspect, a system for encoding a 
frame having a noisy portion is provided. The system 
includes means for determining a macroblock activity 
level and means for determining when the macroblock 

2 0 activity level exceeds a predefined threshold. The 

macroblock activity level exceeding the predefined 
threshold is indicative that the macroblock is 
associated with the noisy portion of the frame. The 
system further includes means for adjusting encoding 
25 of the macroblock when the macroblock activity level 
exceeds the predefined threshold in order to reduce 
bits used in encoding the macroblock, and thereby 
conserve bits otherwise used to encode macroblocks 
within the noisy portion of the frame. 

3 0 In a further aspect, a system is provided for 

encoding a frame of a sequence of frames. This 
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system includes a pre-encode processing unit for 
determining whether the frame includes a random noise 
portion, and a control and encode unit for evaluating 
each macroblock of a plurality of macroblocks 
5 comprising the frame when the frame includes the 
random noise portion. The control and encode unit 
includes means for adjusting encoding of at least 
some macroblocks within the random noise portion of 
the frame to reduce bits used in encoding the 
10 macroblocks within the random noise portion. 

In still other aspects, the concepts presented 
herein are implemented within computer program 
products having computer usable medium with computer 
readable program code means therein for use in 
15 encoding a frame as summarized above. 

Advantageously, processing in accordance with 
the present invention prevents noisy macroblocks or 
blocks with random details from consuming all or most 
of the picture bits, which in turn prevents 
2 0 overproduction of bits before the encoder reaches the 
bottom of the picture. This invention essentially 
directs encode bits from the random, busy macroblocks 
to the simpler, normal macroblocks. Less bits are 
used in the highly active and fine detailed area, 

2 5 thereby providing a more constant picture quality. 

Brief Description of the Drawings 

The above-described objects, advantages and 
features of the present invention, as well as others, 
will be more readily understood from the following 

3 0 detailed description of certain preferred embodiments 



EN998028 



-5- 



of the invention, when considered in conjunction with 
the accompanying drawings in which: 

Fig. 1 shows a flow diagram of a generalized 
MPEG-2 compliant encoder 11, including a discrete 
5 cosine transformer 21, a quantizer 23, a variable 

length coder 25, an inverse quantizer 29, an inverse 
discrete cosine transformer 31, motion compensation 
41, frame memory 42, and motion estimation 43. The 
data paths include the i* picture input 111, 

10 difference data 112, motion vectors 113 (to motion 

compensation 41 and to variable length coder 25) , the 
picture output 121, the feedback picture for motion 
estimation and compensation 131, and the motion 
compensated picture 101. This figure has the 

15 assumptions that the 1 th picture exists in frame 

memory or frame store 42 and that the i + l lh is being 
encoded with motion estimation. 



Fig. 2 illustrates the I, P, and B pictures, 
examples of their display and transmission orders, 
2 0 and forward, and backward motion prediction. 

Fig. 3 illustrates the search from the motion 
estimation block in the current frame or picture to 
the best matching block in a subsequent or previous 
frame or picture. Elements 211 and 211' represent 
25 the same location in both pictures. 

Fig. 4 illustrates the movement of blocks in 
accordance with the motion vectors from their 
position in a previous picture to a new picture, and 
the previous picture's blocks adjusted after using 
30 motion vectors. 
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Fig. 5 depicts one embodiment of a frame of 
contrasted complexity having normal video and noisy 
random video portions to be processed in accordance 
with the adaptive encoding of the present invention. 

5 Fig. 6 shows a generalized encode system 300 in 

accordance with the present invention. System 300 
includes pre-encode statistics analysis 310 to 
determine whether an input picture comprises a 
picture of contrasted complexity and based thereon 
10 whether one or more encoding parameters should be 
varied for individual macroblocks of the picture. 
The modified encoding parameters are used by encode 
engine 32 0 in encoding the individual macroblocks of 
the picture. 

15 Fig. 7 is a flowchart of one embodiment of 

identifying a current frame of a sequence of video 
frames as comprising a frame with a noisy or random 
portion for processing in accordance with the present 
invention . 

2 0 Fig. 8 is a flowchart of one embodiment of 

adaptively encoding a picture having a noisy video 
portion in accordance with the present invention. 

Best Mode for Carrying Out the Invention 

The invention relates, for example, to MPEG 
25 compliant encoders and encoding processes such as 

described in "Information Technology-Generic coding 
of moving pictures and associated audio information: 
Video," Recommendation ITU-T H.262, ISO/IEC 13818-2, 
Draft International Standard, 1994. The encoding 
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functions performed by the encoder include data 
input, spatial compression, motion estimation, 
macroblock type generation, data reconstruction, 
entropy coding, and data output. Spatial compression 
5 includes discrete cosine transformation (DCT) , 
quantization, and entropy encoding. Temporal 
compression includes intensive reconstructive 
processing, such as inverse discrete cosine 
transformation, inverse quantization, and motion 
10 compensation. Motion estimation and compensation are 
used for temporal compression functions. Spatial and 
temporal compression are repetitive functions with 

high computational requirements. 

Further, the invention relates, for example, to 
15 a process for performing spatial and temporal 

compression including discrete cosine transformation, 
quantization, entropy encoding, motion estimation, 
motion compensation, and prediction, and even more 
particularly to a system for accomplishing spatial 

2 0 and temporal compression. 

The first compression step is the elimination of 
spatial redundancy, for example, the elimination of 
spatial redundancy in a still picture of an "I" frame 
picture. Spatial redundancy is the redundancy within 
25 a picture. The MPEG-2 Draft Standard is using a 

block based method of reducing spatial redundancy. 
The method of choice is the discrete cosine 
transformation, and discrete cosine transform coding 
of the picture. Discrete cosine transform coding is 

3 0 combined with weighted scalar quantization and run 

length coding to achieve desirable compression. 
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The discrete cosine transformation is an 
orthogonal transformation . Orthogonal 
transformations, because they have a frequency domain 
interpretation, are filter bank oriented. The 
5 discrete cosine transformation is also localized. 
That is, the encoding process samples on an 8x8 
spatial window which is sufficient to compute 64 
transform coefficients or sub-bands. 

Another advantage of the discrete cosine 
10 transformation is that fast encoding and decoding 

algorithms are available. Additionally, the sub-band 
decomposition of the discrete cosine transformation 
is sufficiently well behaved to allow effective use 
of psychovisual criteria. 

15 After transformation, many of the frequency 

coefficients are zero, especially the coefficients 
for high spatial frequencies. These coefficients are 
organized into a zig-zag or alternate-scanned 
pattern, and converted into run-amplitude (run- level) 

20 pairs. Each pair indicates the number of zero 
coefficients and the amplitude of the non-zero 
coefficient. This is coded in a variable length 
code . 

Motion compensation is used to reduce or even 
25 eliminate redundancy between pictures. Motion 

compensation exploits temporal redundancy by dividing 
the current picture into blocks, for example, 
macroblocks, and then searching in previously 
transmitted pictures for a nearby block with similar 
3 0 content. Only the difference between the current 
block pels and the predicted block pels extracted 
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from the reference picture is actually compressed for 
transmission and thereafter transmitted. 

The simplest method of motion compensation and 
prediction is to record the luminance and 
5 chrominance, i.e., intensity and color, of every 
pixel in an "I" picture, then record changes of 
luminance and chrominance, i.e., intensity and color 
for every specific pixel in the subsequent picture. 
However, this is uneconomical in transmission medium 

10 bandwidth, memory, processor capacity, and processing 
time because objects move between pictures, that is, 
pixel contents move from one location in one picture 
to a different location in a subsequent picture. A 
more advanced idea is to use a previous or subsequent 

15 picture to predict where a block of pixels will be in 
a subsequent or previous picture or pictures, for 
example, with motion vectors, and to write the result 
as "predicted pictures" or "P" pictures. More 
particularly, this involves making a best estimate or 

20 prediction of where the pixels or macroblocks of 

pixels of the i* picture will be in the i-l* or i + l th 
picture. It is one step further to use both 
subsequent and previous pictures to predict where a 
block of pixels will be in an intermediate or "B" 

25 picture. 

To be noted is that the picture encoding order 
and the picture transmission order do not necessarily 
match the picture display order. See Fig. 2. For I- 
P-B systems the input picture transmission order is 
30 different from the encoding order, and the input 

pictures must be temporarily stored until used for 
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encoding. A buffer stores this input until it is 
used. 

For purposes of illustration, a generalized 
flowchart of MPEG compliant encoding is shown in Fig. 
5 1. In the flowchart the images of the i^picture and 
the i+l* picture are processed to generate motion 
vectors. The motion vectors predict where a 
macroblock of pixels will be in a prior and/or 
subsequent picture. The use of the motion vectors is 
10 a key aspect of temporal compression in the MPEG 
standard. As shown in Fig. 1 the motion vectors, 
once generated, are used for the translation of the 
macroblocks of pixels, from the i* picture to the i + l* 
picture . 

15 As shown in Fig. 1, in the encoding process, the 

images of the i^picture and the i + l* picture are 
processed in the encoder 11 to generate motion 
vectors which are the form in which, for example, the 
i+l* and subsequent pictures are encoded and 

2 0 transmitted. An input image 111 of a subsequent 

picture goes to the motion estimation unit 43 of the 
encoder. Motion vectors 113 are formed as the output 
of the motion estimation unit 43. These vectors are 
used by the motion compensation Unit 41 to retrieve 
25 macroblock data from previous and/or future pictures, 
referred to as "reference" data, for output by this 
unit. One output of the motion compensation Unit 41 
is negatively summed with the output from the motion 
estimation unit 43 and goes to the input of the 

3 0 Discrete Cosine Transformer 21. The output of the 

discrete cosine transformer 21 is quantized in a 
quantizer 23. The output of the quantizer 23 is split 
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into two outputs, 121 and 131; one output 121 goes 
to a downstream element 25 for further compression 
and processing before transmission, such as to a run 
length encoder; the other output 131 goes through 
5 reconstruction of the encoded macroblock of pixels 

for storage in frame memory 42. In the encoder shown 
for purposes of illustration, this second output 131 
goes through an inverse quantization 29 and an 
inverse discrete cosine transform 31 to return a 
10 lossy version of the difference macroblock. This data 
is summed with the output of the motion compensation 
unit 41 and returns a lossy version of the original 
picture to the frame memory 42. 

As shown in Fig. 2, there are three types of 
15 pictures. There are "Intra pictures" or "I" pictures 
which are encoded and transmitted whole, and do not 
require motion vectors to be defined. These "I" 
pictures serve as a reference image for motion 
estimation. There are "Predicted pictures" or "P" 
2 0 pictures which are formed by motion vectors from a 
previous picture and can serve as a reference image 
for motion estimation for further pictures. Finally, 
there are "Bidirectional pictures" or "B" pictures 
which are formed using motion vectors from two other 
2 5 pictures, one past and one future, and can not serve 
as a reference image for motion estimation. Motion 
vectors are generated from "I" and "P" pictures, and 
are used to form "P" and "B" pictures. 



One method by which motion estimation is carried 
3 0 out, shown in Fig. 3, is by a search from a 

macroblock 211 of an i* picture throughout a region 
of the next picture to find the best match macroblock 
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213. Translating the macroblocks in this way yields 
a pattern of macroblocks for the i + picture, as 
shown in Fig. 4. In this way the i* picture is 
changed a small amount, e.g., by motion vectors and 
5 difference data, to generate the i+l m picture. What 
is encoded are the motion vectors and difference 
data, and not the i+l* picture itself. Motion vectors 
translate position of an image from picture to 
picture, while difference data carries changes in 
10 chrominance, luminance, and saturation, that is, 
changes in shading and illumination. 

Returning to Fig. 3, we look for a good match by 
starting from the same location in the i th picture as 
in the i + l* picture. A search window is created in 
the i 1 * 1 picture. We search for a best match within 
this search window. Once found, the best match motion 
vectors for the macroblock are coded. The coding of 
the best match macroblock includes a motion vector, 
that is, how many pixels in the y direction and how 
many pixels in the x direction is the best match 
displaced in the next picture. Also encoded is 
difference data, also referred to as the "prediction 
error", which is the difference in chrominance and 
luminance between the current macroblock and the best 
match reference macroblock. 

The operational functions of an MPEG- 2 encoder 
are discussed in detail in commonly assigned, co- 
pending United States Patent Application Serial No. 
08/831,157, by Carr et al . , filed April 1, 1997, 

30 entitled "Control Scheme For Shared-Use Dual-Port 

Predicted Error Array, " which is hereby incorporated 
herein by reference in its entirety. 



20 
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Encoder performance and picture quality are 
often enhanced today through the use of adaptive 
quantization. Examples of adaptive quantization are 
presented in co-pending, commonly assigned United 
5 States Patent Applications by Boroczky et al . , 
entitled "Adaptive Real-Time Encoding of Video 
Sequence Employing Image Statistics," filed October 
10, 1997, serial no. 08/948,442, and by Boice et al., 
entitled "Real-Time Variable Bit Rate Encoding of 
10 Video Sequence Employing Image Statistics," filed 
January 16, 1998, serial no. 09/008,282, both of 
which are hereby incorporated herein by reference in 
their entirety. 

Adaptive quantization can be used to control the 
15 amount of data generated so that an average amount of 
data is output by the encoder and so that this 
average will match a specified bitrate . As one 
approach, video quality of a picture having a noisy 
video portion can be balanced by channeling bits from 
2 0 the noisy or high activity macroblocks to the normal 
portion of the picture. For example, sophisticated 
pre-processing might initially be used to determine 
how picture target bits are to be allocated among all 
the macroblocks of a picture having noisy video. 
25 However, there are 13 50 macroblocks in a NTSC picture 
and 1440 macroblocks in a PAL picture, and the amount 
of preprocessing logic to accomplish this approach 
would require significant buffering and a large 
amount of computational intelligence. 

30 As a preferred approach, presented herein is a 

novel design for dynamically balancing picture bit 
allocation within a highly contrasted picture having 
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normal video and noisy video sections as picture 
coding continues without significant buffering of the 
picture and without requiring large computational 
intelligence to accomplish balancing of the bit 
5 allocation. 

Fig. 5 depicts one embodiment of a picture 250 
of contrasted complexity having a random noise 
portion 260 and a normal video portion 270. As used 
in this application, a "contrasted picture" or 

10 "picture of contrasted complexity" means any picture 
having a first area of high or random activity and a 
second area of significantly lower activity. "Noisy 
video" is used herein to denote a picture or that 
portion of a picture having very high complexity, 

15 such as a picture portion having randomly moving dots 
of different color. "Normal video" is used to mean a 
picture or portion of a picture depicting, for 
example, a conventional motion picture image. Fig. 5 
is thus shown by way of example only and those 

20 skilled in the art will understand that a frame 
having contrasted complexity sections of "normal 
video" and "noisy video" can encompass many 
variations . 

In accordance with this invention, the 
25 complexity of each input picture is statistically 

calculated as the picture is received by the encoder. 
This complexity measurement is tailored to indicate 
the degree of business or amount of detail within the 
picture. From picture complexity, an average 
3 0 complexity value for each macroblock can be 

determined. During the macroblock coding process, 
the encoder calculates the actual macroblock 
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complexity and alters the coding options in 
accordance with this invention when picture 
complexity is above a predefined, experimentally 
determined complexity threshold, and the specified 
5 bitrate is lower than a predefined bitrate threshold. 
The complexity and bitrate thresholds can be selected 
experimentally by one skilled in the art in order to 
accomplish the objects of the present invention. 
Basically, this invention seeks to dynamically modify 
10 the coding algorithm when the bitrate is too low for 
the material to be encoded given that the current 
picture has been statistically determined to comprise 
a picture having a noisy portion of very high 
activity. 

15 Changes to the coding algorithm can include 

adjusting the macroblock coding type and modifying 
the quantization level. For example, once a 
contrasted picture is identified, the macroblock 
coding type is preferably biased towards being coded 

20 predictive, that is, it requires a larger prediction 
error before a macroblock will be coded as intra. 
When the macroblock is coded as intra, the macroblock 
is thus truly different from the prior reference 
picture. Since intra macroblocks take many more bits 

25 to code than predictive macroblocks, the quantization 
level of these macroblocks is also adjusted to 
conserve bits. 

For example, a more precise quantization level 
can be determined from an activity value that is a 
30 better representation of the macroblock to be 

encoded. The relative activity of each block in a 
macroblock is examined, and the block activity that 
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is exceptionally far from the rest is discarded. In 
one embodiment, the block activities can be 
prioritized and the smallest activity value is 
compared to the next smallest one. If the block with 
5 the smallest amount of activity is one-half or less 
the block with the next smallest activity, and is 
one-half or less the average activity within the 
macroblock, then that block with the lowest activity 
is preferably ignored in the quantization level 

10 calculation. The calculated quantization level can 
also be increased by a percentage determined from 
experiments. Again, the goal is to conserve bits 
when encoding macroblocks of the noisy video portion, 
thereby providing more bits for encoding macroblocks 

15 within the normal video portion. 

Fig. 6 depicts one embodiment of an encode 
system, generally denoted 3 00, in accordance with 
this invention. As shown, an input stream of video 
frames is conventionally buffered in frame memory 

20 330. Controller 340 determines where a given input 
picture should be placed within the memory, as well 
as when to encode the picture. While buffered, 
preprocessing of the input stream by statistics 
gathering and analysis 310 is performed in accordance 

25 with the invention. Pre-encode stage 310 gathers and 
analyzes statistics on each frame of the sequence of 
video frames to determine whether the frame has high 
complexity indicative of noisy video and places the 
below-described statistics into a stack 314. 

30 Stacking of input picture statistics is needed 

because the GOP structure employed in MPEG encoding 
of a sequence of video frames may have to be 
reordered prior to encoding. 
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When a given frame is to be encoded, 
preprocessing 310 thus analyzes the frame to 
determine whether one or more encoding parameters 
should be adjusted on a macroblock level. As 
5 described further below, adjustable parameters may 
include macroblock coding type and macroblock 
quantization level. This information is forwarded to 
the encoder engine 32 0 commensurate with retrieval of 
the frame to be compressed from memory 330. Unless 
10 otherwise stated herein, encode engine 320 can 

comprise conventional MPEG compression processing as 
summarized initially herein. 

By way of example, statistics analysis 310 
determines whether the current frame has high 

15 complexity by determining a statistic equal to an 
accumulation of the absolute values of differences 
between pairs of adjacent pixels in the frame. This 
accumulation is referred to herein as "PIX-DIFF" . 
PIX-DIFF can be determined by imagining, for example, 

2 0 the luminance data lines of the current picture 
concatenated to form a long line of luminance 
samples. Then for a given picture, the equation for 
the PIX-DIFF statistic might be: 



25 PIX-DIFF = Z> |L y - L y + 1 

y = 1,3.5 



Where: y is the pixel position number from "1" to the 
maximum number of pixels in the concatenated string 
of pixels. The PIX-DIFF statistic essentially 
3 0 comprises finding the difference between two adjacent 
luminance pixels in this concatenated string of 
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luminance data for the frame and then summing the 
absolute values of those differences. As an 
alternative, PIX-DIFF could be defined as an 
accumulation of both luminance and chrominance data 
5 for the current frame, or an accumulation of 
chrominance data only. 

Fig. 7 depicts one embodiment for statistics 
gathering and analysis in accordance with this 
invention. Upon an input picture being available 

10 500, statistics processing calculates picture 

complexity 510 by determining a PIX-DIFF value for 
the picture. A picture with a noisy portion of 
random detail will have a very high PIX-DIFF value, 
and thus high complexity. The calculated complexity 

15 or PIX-DIFF is compared against an experimentally 
determined, predefined complexity threshold (TH 1) 
520 . 

Applicants have discovered that in measuring the 
PIX-DIFF value for a normal video portion and 

2 0 comparing it to video having a noisy portion, the 

noisy portion has a significantly higher PIX-DIFF 
value. Thus, if the PIX-DIFF for the frame is less 
than the predefined threshold, a noisy picture flag 
is set to "0" 53 0, meaning that the picture comprises 
25 normal video only. However, if the complexity of the 
picture is high (meaning that the frame contains a 
noisy portion) , then the target bitrate for the 
picture is examined. When the bitrate is high (for 
example, 50 Mbits) , there may be sufficient bits to 

3 0 encode even a picture with normal and noisy video 

portions. Conversely, if the bitrate for the frame 
is low, e.g., 4 Mbits, then there may be insufficient 
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bits to adequately encode the frame. Under this 
scenario, the encoding options are preferably 
modified in accordance with this invention. Thus, 
when the bitrate for the frame is greater than a 
5 predefined bitrate threshold (TH 2) , the noisy 

picture flag is set to "0" 53 0 7 and when the bitrate 
is less than this threshold, the noisy picture flag 
is set to "1" 550. The processing of Fig. 7 thus 
results in the setting of a "noisy picture" flag to 
10 either "0" or "1" . In one embodiment, this flag can 
be within the statistics analysis 310 preprocessing 
(Fig. 6) and is accessible by the encoder engine 320 
upon commencement of encoding of the current frame . 

Fig. 8 presents one embodiment for adapting 
15 encoding of a picture having a noisy video portion in 
accordance with the present invention. Picture 
encoding 600 begins by checking whether the noisy 
picture flag (Fig. 7) has been set 610. If the noisy 
picture flag is "0", then normal picture encoding 620 

2 0 is employed. Upon completion of normal picture 

coding, the encode engine returns 63 0 to encode the 
next picture in a sequence of pictures. 

On the other hand, if the noisy picture flag has 
been set, then the macroblock counter is set to "1" 
25 640 and an activity level for each block in the first 
macroblock is determined 650. The four blocks of the 
macroblock are ordered based upon their activity 
level from minimum to maximum and an average block 

activity is determined from the four values. 

3 0 If two times the minimum activity level of the 

blocks is less than the activity level of the next to 
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minimum block in the macroblock, and two times the 
minimum activity level in the macroblock is less than 
the average activity level of the blocks in the 
macroblock, then the macroblock activity is set to a 
5 value equal to the activity level of the next to 
minimum block in the macroblock. Otherwise, the 
macroblock activity is set to the minimum activity 
level in the macroblock 660. 

Once the macroblock activity level is set, it is 
10 compared against a predefined activity threshold (TH 
3) 670. If macroblock activity is below the 
threshold, then normal macroblock coding 680 is 
performed; and processing determines whether the 
macroblock count is at the maximum for the picture 
15 72 0. If not, the macroblock count is incremented 73 0 
and the activity level for the next macroblock in the 
picture is calculated. Otherwise, encode processing 
has been completed, and return is made to process a 
next picture in the sequence 74 0. 

2 0 If the macroblock activity level is greater than 

the predefined activity threshold (TH 3), then motion 
estimation is performed 690 and the prediction error 
or macroblock difference (MBD) is evaluated. If the 
MBD for the macroblock is greater than, for example, 
25 4096 (4k) and 2x (MBD) is greater than the macroblock 
activity level, then the macroblock is coded as an 
intra (I) macroblock 700. Otherwise, the macroblock 
is coded as predictive. Once the coding type is 
determined, the quantization level is calculated 700. 

3 0 The adjusted quantization level is preferably defined 

as : 
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ADJ QL=MIN({1 + 0.25 (TH2 - BR + 1) ) * CAL QL, MAX ALLOWED BY 
STANDARD) 

Where: BR is the target bitrate for the 
macroblock; 

5 TH2 is a predefined bitrate threshold; 

CAL QL is the calculated quantization level 
for the macroblock; and 

MAX ALLOWED BY STANDARD is the maximum 
quantization allowed by MPEG standard. 

10 Essentially, the quantization level is increased in 
order to conserve bits when the macroblock has high 
activity. Once the quantization level is determined, 
it is employed in encoding the macroblock. The 
macroblock count is then evaluated to determine 

15 whether all macroblocks in the picture have been 

encoded, and processing continues as described above. 

Those skilled in the art will note from the 
description provided herein that processing in 
accordance with the present invention prevents noisy 

20 macroblocks or blocks with random details from 

consuming all or most of the picture bits, which in 
turn prevents overproduction of bits before the 
encoder reaches the bottom of the picture. This 
invention essentially directs encoding bits from the 

25 random, busy macroblocks to the simpler, normal 

macroblocks. Less bits are used in the highly active 
and fine detailed area, and thereby a more constant 
picture quality is obtained. 

The present invention can be included, for 
30 example, in an article of manufacture (e.g., one or 
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more computer program products) having, for instance, 
computer usable media. This media has embodied 
therein, for instance, computer readable program code 
means for providing and facilitating the capabilities 
5 of the present invention. The articles manufactured 
can be included as part of the computer system or 
sold separately. 

The flow diagrams depicted herein are provided 
by way of example. There may be variations to these 

10 diagrams or the steps or operations described herein 
without departing from the spirit of the invention. 
For instance, in certain cases the steps may be 
performed in differing order, or steps may be added, 
deleted or modified. All these variations are 

15 considered to comprise part of the present invention 
as recited in the appended claims. 

While the invention has been described in detail 
herein in accordance with certain preferred 
embodiments thereof, many modifications and changes 
2 0 therein may be affected by those skilled in the art. 
Accordingly, it is intended by the appended claims to 
cover all such modifications and changes as fall 
within the true spirit and scope of the invention. 
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Claims 



1 1. A method for encoding a frame having a 

2 noisy portion, said frame comprising a plurality of 

3 macroblocks, said method comprising for each 

4 macroblock of said plurality of macroblocks: 

5 (i) determining a macroblock activity 

6 level; 

7 (ii) determining when said macroblock 

8 activity level exceeds a predefined threshold, 

9 wherein said macroblock activity level exceeding 

10 said predefined threshold indicates that said 

11 macroblock is associated with said noisy portion 

12 of said frame; and 

13 (iii) adjusting encoding of said 

14 macroblock when said macroblock activity level 

15 exceeds said predefined threshold to conserve 

16 bits used in encoding said macroblock and 

17 thereby save bits otherwise used to encode said 

18 noisy portion of said frame. 

1 2. The method of claim 1, wherein said frame 

2 further comprises a normal portion, and wherein said 

3 method comprises using said saved bits from said 

4 noisy portion of said frame to encode macroblocks 

5 associated with said normal portion of said frame. 
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1 3. The method of claim 1, wherein each 

2 macroblock of said plurality of macroblocks comprises 

3 multiple blocks, and wherein said determining (i) 

4 comprises determining an activity level for each 

5 block of said multiple blocks of said macroblock, and 

6 deriving therefrom an activity level for said 

7 macroblock. 

1 4. The method of claim 3, wherein said 

2 deriving comprises ordering activity levels of said 

3 multiple blocks of said macroblock and comparing a 

4 minimum activity level of said order with a next to 

5 minimum activity level of said order to derive said 

6 activity level for said macroblock. 

1 5. The method of claim 4, wherein said 

2 comparing further comprises comparing said minimum 

3 activity level of said order with an average activity 

4 level of said multiple blocks of said macroblock to 

5 derive said activity level for said macroblock. 

1 6. The method of claim 5, wherein said 

2 comparing comprises determining whether said minimum 

3 activity level is less than one-half said next to 

4 minimum activity level and whether said minimum 

5 activity level is less than one-half said average 

6 activity level of said multiple blocks, and when both 

7 are so, defining said activity level of said 

8 macroblock as said next to minimum activity level of 

9 said order, otherwise defining said activity level of 

10 said macroblock as said minimum activity level of 

11 said order. 
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1 7. The method of claim 1, wherein said 

2 adjusting encoding (iii) comprises performing motion 

3 estimation on said macroblock and selectively 

4 adjusting macroblock coding type for said macroblock 

5 to bias said macroblock towards being coded 

6 predictive when said macroblock activity level 

7 exceeds said predefined threshold, said selectively 

8 adjusting being with reference to a predictive error 

9 value resulting from said performing motion 
10 estimation on said macroblock. 

1 8. The method of claim 7 , wherein said 

2 selectively adjusting comprises determining when said 

3 predictive error is greater than a second predefined 

4 threshold and said predictive error is greater than 

5 one -half said macroblock activity level, and when 

6 both are so, adjusting a macroblock coding type 

7 parameter to bias said macroblock towards being coded 

8 predictive . 

1 9. The method of claim 1, wherein said 

2 adjusting encoding (iii) comprises determining an 

3 adjusted quantization level for use in encoding said 

4 macroblock, said adjusted quantization level being 

5 determined to conserve bits used in encoding said 

6 macroblock when said macroblock activity level 

7 exceeds said predefined threshold. 



EN998028 



-26- 



1 10. The method of claim 9, wherein said 

2 determining of said adjusted quantization level 

3 comprises calculating a quantization level (CAL QL) 

4 for said macroblock and defining said adjusted 

5 quantization level (ADJ QL) as: 

6 ADJ QL=MIN((1 + 0.25 (TH2 - BR + 1) ) * CAL QL; MAX ALLOWED BY 

7 STANDARD) 

8 Where: BR is the target bitrate; 

9 TH2 is a second predefined value; and 

10 MAX QL ALLOWED BY STANDARD is a maximum 

11 quantization level allowed by MPEG standard. 

1 11. The method of claim 1, wherein said frame 

2 comprises one frame of a sequence of frames, and said 

3 method further comprises initially determining for 

4 each frame of said sequence of frames whether said 

5 frame includes said noisy portion. 

1 12. The method of claim 11, wherein said 

2 determining whether said frame comprises said noisy 

3 portion includes calculating a frame complexity value 

4 and comparing said frame complexity value to a 

5 predefined complexity threshold. 

1 13. The method of claim 12, wherein said frame 

2 comprises a plurality of pixels, and wherein each 

3 pixel of said frame comprises a mult i -bit value, and 

4 wherein said frame complexity value comprises an 

5 accumulated absolute difference value (PIX-DIFF) 

6 derived from adjacent pixels of said plurality of 

7 pixels of said frame. 
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1 14. The method of claim 13, wherein said PIX- 

2 DIFF is defined as: 



E 



4 |L y - L y+1 | 

5 y=l,3,5. . 

6 Where: L represents luminance value of a pixel, 

7 and y represents pixel position within the 

8 frame . 

1 15. The method of claim 13, further comprising 

2 setting a noisy picture flag to "0" when said frame 

3 complexity value is less than said predefined 

4 complexity threshold. 

1 16. The method of claim 13, wherein said 

2 determining whether said frame comprises said noisy 

3 portion further includes comparing a target bitrate 

4 for said frame to a predefined bitrate threshold and 

5 when said target bitrate for said frame exceeds said 

6 predefined bitrate threshold, said method further 

7 comprises setting a noisy picture flag equal to "0", 

8 and if said target bitrate is less than said 

9 predefined bitrate threshold, then setting said noisy 

10 picture flag to "1" , wherein said "1" noisy picture 

11 flag setting indicates said frame includes said noisy 

12 portion. 
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1 17. A method for encoding a frame of a sequence 

2 of frames, each frame having a plurality of 

3 macroblocks, said method comprising: 

4 determining whether said frame includes a 

5 random noise portion; and 

6 when said frame includes said random noise 

7 portion, evaluating each macroblock of said 

8 plurality of macroblocks in said frame and 

9 adjusting encoding of at least some macroblocks 

10 thereof within said random noise portion of said 

11 frame, said adjusting comprising reducing bits 

12 used in encoding said at least some macroblocks 

13 within said random noise portion. 

1 18. The method of claim 17, wherein each frame 

2 of the sequence of frames comprises a plurality of 

3 pixels, each pixel of each frame comprising a multi- 

4 bit value, and wherein said determining whether said 

5 frame includes said random noise portion includes 

6 calculating a frame complexity value and comparing 

7 said frame complexity value to a predefined 

8 complexity threshold, said calculating of said frame 

9 complexity value including deriving an accumulated 

10 absolute difference (PIX-DIFF) from adjacent pixels 

11 of said plurality of pixels of said frame. 
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1 19. The method of claim 18, wherein said 

2 deriving of said PIX-DIFF comprises forming a string 

3 of pixels by concatenating said plurality of pixels 

4 of said frame and defining PIX-DIFF as: 

5 Max 

6 £ |Ly - L y+1 | 

7 

7 y= 1.3.5 . 

8 Where: L represents luminance value of a pixel, 

9 and y represents pixel position within the 
10 string of pixels. 

1 20. The method of claim 18, wherein when said 

2 frame complexity value is less than said predefined 

3 complexity threshold, said method further comprises 

4 setting a noisy picture flag to "0" and performing 

5 normal encoding on said frame, and wherein when said 

6 frame complexity value is greater than said 

7 predefined complexity threshold, said method further 

8 comprises determining whether a target bitrate of 

9 said frame is less than a predefined bitrate 

10 threshold, wherein when said target bitrate of said 

11 frame exceeds said predefined bitrate threshold, said 

12 method comprises setting said noisy picture flag to 

13 "0", and when said target bitrate of said frame is 

14 less than said predefined bitrate threshold, said 

15 method comprises setting said noisy picture flag to 

16 "1", wherein said "1" noisy picture flag setting 

17 indicates that said frame includes said random noise 

18 portion. 
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1 21. The method of claim 17, wherein said 

2 evaluating comprises for each macroblock determining 

3 a macroblock activity level and determining when said 

4 macroblock activity level exceeds a predefined 

5 activity threshold, wherein said macroblock activity 

6 level exceeding said predefined activity threshold 

7 indicates that said macroblock is within said random 

8 noise portion of said frame. 

1 22. The method of claim 21, wherein said 

2 adjusting encoding comprises performing motion 

3 estimation on said macroblock and selectively 

4 adjusting macroblock coding type for said macroblock 

5 to bias said macroblock towards being coded 

6 predictive when said macroblock activity level 

7 exceeds said predefined activity threshold, said 

8 selectively adjusting being with reference to a 

9 predictive error value resulting from said performing 

10 motion estimation on said macroblock, and further 

11 comprising determining an adjusted quantization level 

12 for said macroblock for use in encoding said 

13 macroblock, said adjusted quantization level being 

14 determined to reduce bits used in encoding said 

15 macroblock. 

1 23. The method of claim 17, wherein said frame 

2 further includes a normal video portion, and said 

3 reducing bits comprises conserving bits used in 

4 encoding said at least some macroblocks within said 

5 random noise portion for use within said normal video 

6 portion of said frame. 
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1 24 . A system for encoding a frame having a 

2 noisy portion, said frame comprising a plurality of 

3 macroblocks, said system comprising: 

4 (i) means for determining a macroblock 

5 activity level; 

6 (ii) means for determining when said 

7 macroblock activity level exceeds a predefined 

8 threshold, wherein said macroblock activity 

9 level exceeding said predefined threshold 

10 indicates that said macroblock is associated 

11 with said noisy portion of said frame; and 

12 (iii) means for adjusting encoding of said 

13 macroblock when said macroblock activity level 

14 exceeds said predefined threshold to conserve 

15 bits used in encoding said macroblock and 

16 thereby save bits otherwise used to encode said 

17 noisy portion of said frame. 

1 25. The system of claim 24, wherein said frame 

2 further comprises a normal portion, and wherein said 

3 system comprises means for using said saved bits from 

4 said noisy portion of said frame to encode 

5 macroblocks associated with said normal portion of 

6 said frame. 
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1 26. The system of claim 24, wherein each 

2 macroblock of said plurality of macroblocks comprises 

3 multiple blocks, and wherein said means for 

4 determining (i) comprises means for determining an 

5 activity level for each block of said multiple blocks 

6 of said macroblock, and means for ordering activity 

7 levels of said multiple blocks of said macroblock and 

8 comparing a minimum activity level of said order with 

9 a next to minimum activity level of said order to 
10 derive an activity level for said macroblock. 

1 27. The system of claim 26, wherein said means 

2 for comparing comprises means for determining whether 

3 said minimum activity level is less than one-half 

4 said next to minimum activity level and whether said 

5 minimum activity level is less than one-half an 

6 average activity level of said multiple blocks, and 

7 when both are true, for defining said activity level 

8 of said macroblock as said next to minimum activity 

9 level in said macroblock, otherwise for defining said 

10 activity level of said macroblock as said minimum 

11 activity level of said order. 

1 28. The system of claim 24, wherein said means 

2 for adjusting encoding (iii) comprises means for 

3 performing motion estimation on said macroblock and 

4 for selectively adjusting macroblock coding type for 

5 said macroblock to bias said macroblock towards being 

6 coded predictive when said macroblock activity level 

7 exceeds said predefined threshold, said selectively 

8 adjusting being with reference to a predictive error 

9 value resulting from said performing of motion 
10 estimation on said macroblock. 
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1 29. The system of claim 28, wherein said means 

2 for selectively adjusting comprises means for 

3 determining when said predictive error is greater 

4 than a second predefined threshold and when said 

5 predictive error is greater than one-half said 

6 macroblock activity level, and when both are so, said 

7 means for selectively adjusting comprises means for 

8 adjusting a macroblock coding type parameter to bias 

9 said macroblock towards being coded predictive. 

1 30. The system of claim 29, wherein said means 

2 for adjusting encoding (iii) further comprises means 

3 for determining an adjusted quantization level for 

4 use in encoding said macroblock, said adjusted 

5 quantization level being determined to conserve bits 

6 used in encoding said macroblock when said macroblock 

7 activity level exceeds said predefined threshold. 
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1 31. A system for encoding a frame of a sequence 

2 of frames, each frame having a plurality of 

3 macroblocks, said system comprising: 

4 a pre-encode processing unit for 

5 determining whether said frame includes a random 

6 noise portion; and 

7 a control and encode unit for evaluating 

8 each macroblock of said plurality of macroblocks 

9 in said frame when said frame includes said 

10 random noise portion, said control and encode 

11 unit including means for adjusting encoding of 

12 at least some macroblocks within said random 

13 noise portion of said frame to reduce bits used 

14 in encoding said at least some macroblocks 

15 within said random noise portion. 

1 32. The system of claim 31, wherein each frame 

2 of the sequence of frames comprises a plurality of 

3 pixels, each pixel of each frame comprising a multi- 

4 bit value, and wherein said pre-encode processing 

5 unit comprises means for deriving a frame complexity 

6 value and for comparing said frame complexity value 

7 to a predefined complexity threshold, said means for 

8 deriving of said frame complexity value including 

9 means for deriving an accumulated absolute difference 

10 (PIX-DIFF) from adjacent pixels of said plurality of 

11 pixels of said frame. 
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1 33. The system of claim 32, wherein when said 

2 frame complexity value is less than said predefined 

3 complexity threshold, said pre-encode processing unit 

4 further comprises means for setting a noisy picture 

5 flag to "0" and performing normal encoding on said 

6 frame, and when said frame complexity value is 

7 greater than said predefined complexity threshold, 

8 said pre-encode processing unit comprises means for 

9 determining whether a target bitrate of said frame is 

10 less than a predefined bitrate threshold, and when 

11 said target bitrate of said frame exceeds said 

12 predefined bitrate threshold, said pre-encode 

13 processing unit comprises means for setting said 

14 noisy picture flag to "0", and when said target 

15 bitrate of said frame is less than said predefined 

16 bitrate threshold, said pre-encode processing unit 

17 comprises means for setting said noisy picture flag 

18 to "1", wherein said "1" noisy picture flag setting 

19 indicates that said frame includes said random noise 

20 portion. 

1 34. The system of claim 33, wherein said 

2 control and encode unit further comprises means for 

3 determining for each macroblock a macroblock activity 

4 level and for determining when said macroblock 

5 activity level exceeds a predefined activity 

6 threshold, wherein said macroblock activity level 

7 exceeding said predefined activity threshold 

8 indicates that said macroblock is within said random 

9 noise portion of said frame. 



EN998028 



-36- 



1 35. The system of claim 34, wherein said means 

2 for adjusting encoding comprises means for performing 

3 motion estimation on said macroblock and means for 

4 selectively adjusting macroblock coding type for said 

5 macroblock to bias said macroblock towards being 

6 coded predictive when said macroblock activity level 

7 exceeds said predefined activity threshold, said 

8 means for selectively adjusting being with reference 

9 to a predictive error value resulting from performing 

10 motion estimation on said macroblock, and wherein 

11 said control and encode unit further comprises means 

12 for determining an adjusted quantization level for 

13 said macroblock for use in encoding said macroblock, 

14 said adjusted quantization level being determined to 

15 reduce bits used in encoding said macroblock. 

1 36. The system of claim 35, wherein said frame 

2 further includes a normal video portion, and said 

3 means for adjusting encoding comprises means for 

4 conserving bits used in encoding said at least some 

5 macroblocks within said random noise portion for use 

6 in encoding macroblocks within said normal video 

7 portion of said frame. 
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1 37. A computer program product comprising a 

2 computer usable medium having computer readable 

3 program code means therein for use in encoding a 

4 frame having a noisy portion, said frame comprising a 

5 plurality of macroblocks, said computer readable 

6 program code means in said computer program product 

7 comprising for each macroblock of said plurality of 

8 macroblocks : 

9 computer readable program code means for 

10 causing a computer to affect determining a 

11 macroblock activity level; 

12 computer readable program code means for 

13 causing a computer to affect determining when 

14 said macroblock activity level exceeds a 

15 predefined threshold, wherein said macroblock 

16 activity level exceeding said predefined 

17 threshold indicates that said macroblock is 

18 associated with said noisy portion of said 

19 frame; and 

2 0 computer readable program code means for 

21 causing a computer to affect adjusting encoding 

22 of said macroblock when said macroblock activity 

23 level exceeds said predefined threshold to 

24 conserve bits used in encoding said macroblock 

25 and thereby save bits otherwise used to encode 

26 said noisy portion of said frame. 
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1 3 8. A computer program product comprising 

2 computer usable medium having computer readable 

3 program code means therein for use in encoding a 

4 frame of a sequence of frames, each frame having a 

5 plurality of macroblocks , said computer readable 

6 program code means in said computer program product 

7 comprising: 

8 computer readable program code means for 

9 causing a computer to affect determining whether 

10 said frame includes a random noise portion; and 

11 computer readable program code means for 

12 causing a computer to affect evaluating each 

13 macroblock of said plurality of macroblocks in 

14 said frame and when said frame includes said 

15 random noise portion, adjusting encoding of at 

16 least some macroblocks within said random noise 

17 portion of said frame, said adjusting comprising 

18 reducing bits used in encoding said at least 

19 some macroblocks within said random noise 

20 portion. 



* * * * * 
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ADAPTIVELY ENCODING A PICTURE OF 
CONTRASTED COMPLEXITY HAVING NORMAL 
VIDEO AND NOISY VIDEO PORTIONS 



Abstract of the Disclosure 

5 A technique is provided for adaptively encoding 

in hardware, software or a combination thereof a 
sequence of frames in real time, wherein one or more 
of the frames includes a random noise portion. The 
technique includes using statistics analysis to 

10 determine whether a current frame includes a random 

noise portion, and if so, to evaluate and dynamically 
encode each macroblock thereof based on activity 
level of the macroblock. Evaluating macroblock 
activity level includes determining whether its 

15 activity level exceeds a predefined threshold 
indicative of random noise. The macroblock is 
adaptively encoded by adjusting one or more coding 
parameters if the macroblock activity level is 
excessive and its target bitrate is low. For 

20 example, when the macroblock is within the random 

noise portion of the frame, the macroblock is biased 
towards being coded predictive and an adjusted 
quantization level is calculated to conserve bits 
used in encoding the macroblock, thereby moving 

2 5 encode bits from macroblocks within the random noise 
portion of the frame to macroblocks within the normal 
portion of the frame. 
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Declaration and Power of Attorney For Patent Application 



As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to my name, 

I believe I am the original, first and sole inventor (if only one name is listed below) or an original, 
first and joint inventor (if plural names are listed below) of the subject matter which is claimed and for 
which a patent is sought on the invention entitled 

ADAPTIVELY ENCODING A PICTURE OF CONTRASTED COMPLEXITY HAVING NORMAL VIDEO 
AND NOISY VIDEO PORTIONS 

the specification of which 

^ (check one) 

^ *) is attached hereto. 

:p U was filed on as United States Application No. or PCT International 

Application Number 

and was amended on 



O I hereby state that I have reviewed and understand the contents of the above identified specification, 
Lu including the claims, as amended by any amendment referred to above. 

^ I acknowledge the duty to disclose to the United States Patent and Trademark Office all information 
% known to me to be material to patentability as defined in Title 37, Code of Federal Regulations, 
u * Section 1.56. 

I hereby claim foreign priority benefits under Title 35, United States Code, Section 119(a)-(d) or 
Section 365(b) of any foreign application(s) for patent or inventor's certificate, or Section 365(a) of 
any PCT International application which designated at least one country other than the United 
States, listed below and have also identified below, by checking the box, any foreign application for 
patent or inventor's certificate or PCT International application having a filing date before that of the 
application on which priority is claimed. 

Prior Foreign Application(s) Priority Not Claimed 



English Language Declaration 



(if applicable) 



NONE 



□ 



(Number) 



(Country) 



(Day/Month/Year Filed) 



J 



(Number) 



(Country) 



(Day/Month/Year Filed) 



(Number) 



(Country) 



(Day/Month/Year Filed) 
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I hereby claim the benefit under 35 U.S.C. Section 119(e) of any United States provisional 
application(s) listed below: 

NONE 

(Application Serial No.) (Filing Date) 



(Application Serial No.) (Filing Date) 



(Application Serial No ) (Filing Date) 

I hereby claim the benefit under 35 U S. C. Section 120 of any United States application^), or 
Section 365(c) of any PCT International application designating the United States, listed below and, 
insofar as the subject matter of each of the claims of this application is not disclosed in the prior 
United States or PCT International application in the manner provided by the first paragraph of 35 
U.S.C. Section 1 12, I acknowledge the duty to disclose to the United States Patent and Trademark 
■r% Office all information known to me to be material to patentability as defined in Title 37, C. F. R., 
3 Section 1 .56 which became available between the filing date of the prior application and the national 
O or PCT International filing date of this application: 



(Status) 

(patented, pending, abandoned) 



(Filing Date) (Status) 

(patented, pending, abandoned) 



(Application Serial No.) (Filing Date) (Status) 

(patented, pending, abandoned) 

I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements 
were made with the knowledge that willful false statements and the like so made are punishable by 
fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that 
such willful false statements may jeopardize the validity of the application or any patent issued 
thereon. 



NONE 

(Application Serial No.) (Filing Date) 



(Application Serial No.) 
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POWER OF ATTORNEY* As a named inventor, I hereby appoint the following attorney(s) and/or 
agent(s) to prosecute this application and transact all business in the Patent and Trademark Office 
connected therewith (list name and registration number) 

Christopher A. Hughes, Esq., Reg* No. 26,914 Blanche E. Schiller, Esq., Reg. No. 35,670 

Ednard A. Pennington, Esq., Reg. No. 32,588 

John E. Hocl, Esq., Reg. No. 26,279 

Joseph C. Redmond, Jr., Esq., Reg. No. 18,753 

Da\id L. Adour, Esq., Reg. No. 29,604 

Lawrence R. Fralcv, Esq., Reg. No. 26,885 

Arthur J. Samodo\it/, Esq., Reg. No. 31,297 

Bernard Ticgerman, Esq., Reg. No. 29,707 

John R. Pivnichny, Ph.D., P-43,001 

Jeff Rothcnberg, Esq., Reg. No. 26,429 

Kevin P. Radigan, Esq., Reg. No. 31,789 



Send Correspondence to: Ke " n p * Radi * an > Es <** 

HESLIN & ROTHENBERG, P.C. 

5 Columbia Circle 

Albany, NY 12203-5160 



Direct Telephone Calls to: (name and telephone number) 
Kevin P. Radigan, (518) 452-5600 / Fax: (518) 452-5579 



Full name of sole or first inventor 
Barbara A. Hall 

Sole or first inventor's signature An Date 

6oZl^a,jjJ$ - —tir.-ast. 

Residence 

607 Winston Dr., EnducIS, NY 13760 

Citizenship 

U.S.A. 

Post Office Address 

same as residence 



\ Full name of second inventor, if any 

! Agnes Y. Ngai 

I Second mventoPs signature Date 

mb-JUm ----- ----- — ----- — lHJf- 

Residence 7j Q 

725 Partridge Place, End* ell, NY 13760 
Citizenship 

U.S.A. _ _ _ 

Post Office Address 
same as residence 
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Full name of third inventor, if any 
Charles J. Stein 

Third inventor's signature 
Residence ' 

107 Holly Lane, Peck>illc, PA 18452 

Citizenship 

U.S.A. 



Date 

3-11' H 



Post Office Address 
same as residence 



Full name of fourth inventor, if any 
Everett G. Vail III 

Fourth inventor's signature a 
Residence 

4 Ellen Street, Binghamton, NY 13901 

Citizenship 

U.S.A. 

Post Office Address 
same as residence 



Full name of fifth inventor, if any 
Edward F. Westcrmann 




Res-dene^ * " " 
189 Dorothy Street, Endicott, NY 13760 

Citizenship 
U.S.A. 

Post Office Address 
same as residence 



Date 



Date 

3- 



Lname of sixth inventor, if any 
Sixth inventor's signatufi 
Residence 



Date 



Citizenship 

Post Office Address 
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